Now that we have obtained our data, we will take our first looks into it, seeking out missing values and choosing which parts of the data set will be useful for our classifier. We will also begin pre-processing the text data with natuaral language tools. This section concludes with some exploratory data analysis and visualisations.
Cutting down the dataset - As both sets have 100 columns, it'll be wise to choose a few columns that will be useful as our predictors.
Concatenation - As we have already created a "is_suicide" column indicating which subreddit the posts are from, we should concatenate both datasets together.
Imputation - If there are missing values, we should find a way to impute the data.
| approved_at_utc | subreddit | selftext | author_fullname | saved | mod_reason_title | gilded | clicked | title | link_flair_richtext | subreddit_name_prefixed | hidden | pwls | link_flair_css_class | downs | hide_score | name | quarantine | link_flair_text_color | author_flair_background_color | subreddit_type | ups | total_awards_received | media_embed | author_flair_template_id | is_original_content | user_reports | secure_media | is_reddit_media_domain | is_meta | category | secure_media_embed | link_flair_text | can_mod_post | score | approved_by | author_premium | thumbnail | edited | author_flair_css_class | author_flair_richtext | gildings | content_categories | is_self | mod_note | created | link_flair_type | wls | removed_by_category | banned_by | author_flair_type | domain | allow_live_comments | selftext_html | likes | suggested_sort | banned_at_utc | view_count | archived | no_follow | is_crosspostable | pinned | over_18 | all_awardings | awarders | media_only | can_gild | spoiler | locked | author_flair_text | visited | removed_by | num_reports | distinguished | subreddit_id | mod_reason_by | removal_reason | link_flair_background_color | id | is_robot_indexable | report_reasons | author | discussion_type | num_comments | send_replies | whitelist_status | contest_mode | mod_reports | author_patreon_flair | author_flair_text_color | permalink | parent_whitelist_status | stickied | url | subreddit_subscribers | created_utc | num_crossposts | media | is_video | is_suicide | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | NaN | depression | We understand that most people who reply immed... | t2_1t70 | False | NaN | 0 | False | Our most-broken and least-understood rules is ... | [] | r/depression | False | 0.0 | NaN | 0 | False | t3_doqwow | False | dark | NaN | public | 1818 | 0 | {} | NaN | False | [] | NaN | False | False | NaN | {} | NaN | False | 1818 | NaN | True | NaN | False | NaN | [] | {} | NaN | True | NaN | 1.572390e+09 | text | 0.0 | NaN | NaN | text | self.depression | False | <!-- SC_OFF --><div class="md"><... | NaN | confidence | NaN | NaN | False | False | False | False | False | [] | [] | False | False | False | False | NaN | False | NaN | NaN | moderator | t5_2qqqf | NaN | NaN | NaN | doqwow | True | NaN | SQLwitch | NaN | 133 | True | no_ads | False | [] | False | NaN | /r/depression/comments/doqwow/our_mostbroken_a... | no_ads | True | https://www.reddit.com/r/depression/comments/d... | 611580 | 1.572361e+09 | 0 | NaN | False | 0 |
| 1 | NaN | depression | Welcome to /r/depression's check-in post - a p... | t2_64qjj | False | NaN | 0 | False | Regular Check-In Post | [] | r/depression | False | 0.0 | NaN | 0 | False | t3_exo6f1 | False | dark | NaN | public | 310 | 0 | {} | NaN | False | [] | NaN | False | False | NaN | {} | NaN | False | 310 | NaN | False | NaN | False | NaN | [] | {} | NaN | True | NaN | 1.580678e+09 | text | 0.0 | NaN | NaN | text | self.depression | True | <!-- SC_OFF --><div class="md"><... | NaN | new | NaN | NaN | False | False | False | False | False | [] | [] | False | False | False | False | NaN | False | NaN | NaN | moderator | t5_2qqqf | NaN | NaN | NaN | exo6f1 | True | NaN | circinia | NaN | 1644 | False | no_ads | False | [] | False | NaN | /r/depression/comments/exo6f1/regular_checkin_... | no_ads | True | https://www.reddit.com/r/depression/comments/e... | 611580 | 1.580649e+09 | 0 | NaN | False | 0 |
| 2 | NaN | depression | I've been feeling really depressed and lonely ... | t2_17aooz | False | NaN | 0 | False | I hate it so much when you try and express you... | [] | r/depression | False | 0.0 | NaN | 0 | False | t3_fedwbi | False | dark | NaN | public | 89 | 0 | {} | NaN | False | [] | NaN | False | False | NaN | {} | NaN | False | 89 | NaN | False | NaN | False | NaN | [] | {} | NaN | True | NaN | 1.583532e+09 | text | 0.0 | NaN | NaN | text | self.depression | False | <!-- SC_OFF --><div class="md"><... | NaN | confidence | NaN | NaN | False | False | False | False | False | [] | [] | False | False | False | False | NaN | False | NaN | NaN | NaN | t5_2qqqf | NaN | NaN | NaN | fedwbi | True | NaN | TheNewKiller69 | NaN | 8 | True | no_ads | False | [] | False | NaN | /r/depression/comments/fedwbi/i_hate_it_so_muc... | no_ads | False | https://www.reddit.com/r/depression/comments/f... | 611580 | 1.583503e+09 | 0 | NaN | False | 0 |
| 3 | NaN | depression | I literally broke down crying and asked to go ... | t2_5v2j4itq | False | NaN | 0 | False | I went to the hospital because I was having re... | [] | r/depression | False | 0.0 | NaN | 0 | False | t3_feel0k | False | dark | NaN | public | 39 | 0 | {} | NaN | False | [] | NaN | False | False | NaN | {} | NaN | False | 39 | NaN | False | NaN | False | NaN | [] | {} | NaN | True | NaN | 1.583535e+09 | text | 0.0 | NaN | NaN | text | self.depression | False | <!-- SC_OFF --><div class="md"><... | NaN | confidence | NaN | NaN | False | False | False | False | False | [] | [] | False | False | False | False | NaN | False | NaN | NaN | NaN | t5_2qqqf | NaN | NaN | NaN | feel0k | True | NaN | Jazzlecrab | NaN | 15 | True | no_ads | False | [] | False | NaN | /r/depression/comments/feel0k/i_went_to_the_ho... | no_ads | False | https://www.reddit.com/r/depression/comments/f... | 611580 | 1.583507e+09 | 0 | NaN | False | 0 |
| 4 | NaN | depression | Any kind soul want to give a depressed person ... | t2_15xfmv | False | NaN | 0 | False | Cake day for me | [] | r/depression | False | 0.0 | NaN | 0 | False | t3_fe6ua3 | False | dark | NaN | public | 315 | 2 | {} | NaN | False | [] | NaN | False | False | NaN | {} | NaN | False | 315 | NaN | False | NaN | 1583471814.0 | NaN | [] | {'gid_1': 2} | NaN | True | NaN | 1.583491e+09 | text | 0.0 | NaN | NaN | text | self.depression | False | <!-- SC_OFF --><div class="md"><... | NaN | confidence | NaN | NaN | False | False | False | False | False | [{'count': 2, 'is_enabled': True, 'subreddit_i... | [] | False | False | False | False | NaN | False | NaN | NaN | NaN | t5_2qqqf | NaN | NaN | NaN | fe6ua3 | True | NaN | Depressed_Kid786 | NaN | 37 | True | no_ads | False | [] | False | NaN | /r/depression/comments/fe6ua3/cake_day_for_me/ | no_ads | False | https://www.reddit.com/r/depression/comments/f... | 611580 | 1.583463e+09 | 0 | NaN | False | 0 |
Title and Post - We felt that the text data in the both the title and the post itself can potentially serve our classifier well.
Author's handle and number of comments - The author's name and the number of comments are curve ball choices. There just might be some connection between a user's handle and his/her psyche. There also might be a connection between the number of comments made.
URL - We left the URL in for reference. In case we'd want to look deeper into a particular post.
| title | selftext | author | num_comments | is_suicide | url | |
|---|---|---|---|---|---|---|
| 0 | Our most-broken and least-understood rules is ... | We understand that most people who reply immed... | SQLwitch | 133 | 0 | https://www.reddit.com/r/depression/comments/d... |
| 1 | Regular Check-In Post | Welcome to /r/depression's check-in post - a p... | circinia | 1644 | 0 | https://www.reddit.com/r/depression/comments/e... |
| 2 | I hate it so much when you try and express you... | I've been feeling really depressed and lonely ... | TheNewKiller69 | 8 | 0 | https://www.reddit.com/r/depression/comments/f... |
| title | selftext | author | num_comments | is_suicide | url | |
|---|---|---|---|---|---|---|
| 0 | New wiki on how to avoid accidentally encourag... | We've been seeing a worrying increase in pro-s... | SQLwitch | 260 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 1 | Reminder: Absolutely no activism of any kind i... | If you want to recognise an occasion, please d... | SQLwitch | 124 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 2 | To every single poster here i wanne say one thing | I really fucking feel you | NussNougatCroissant | 46 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
(917, 6)
(980, 6)
Hello. Depression has always been a secondary problem for me, with my main antagonist being severe Harm OCD. But since my relationship ended 8 months ago, I've been stuck in this horrific cycle of absolutely loathing myself, feeling heavy/tired and totally unmotivated to do anything. It's like I'm living in a 2 dimensional world. Nothing in life jumps out and catches my attention like it used to. I used to be quite creative but it's just taken a nose dive. Any work I do is utterly awful and I'm amazed I'm not been kicked off projects (I work freelance). I wake up and I just want to be dead, quite honestly. In fact in the last few weeks I've even found getting out of bed to be a monumental struggle in itself, where I'm almost in tears from the weight of everything. I am getting help with a therapist and am on an SSRI/low dose anti psych. And I've taken their advice seriously in terms of behavioural activation and avoiding staying in bed. I've started cycling again (just done 20 miles today in fact). But I still feel utterly wretched and everything I do is laced with this horrible belief that this is the end and suicide is my only way out of the pain I'm going through. The years ahead fill me with dread (I'm in my 30s). With my Harm OCD I have a huge hatred of myself anyway, but lately things have just got so heavy. I'd just like to know if anybody can relate to my post and has any first person advice? Psychiatrists and therapists always feel so removed from the reality (for me, anyway) - I'd just like to hear from people who know what's like and have helped themselves.
1595
I'm an 18 year old with severe depression, anxiety, ADHD, borderline personality disorder and DID at a point in my life. But I'm slowly recovering. I have always felt like I never belonged to this world, because I am "too imaginative" and "too kind". I behave "not like how people should". I'm mostly well-liked, but I know they think I'm a weirdo. Everyone does. It's final exam time for us, and I have to secure good marks, otherwise I won't be able to enter any good streams in a good college. It would be the end of my future. My family is in a financial crunch, but they give up everything for me. I am dead-set on helping my friends get through their depression and suicidal tendencies, but I'm failing at that. I have had three attempts, but I was saved by a person, who himself was alexithymic (I think) and suicidal. He still is, and he says if he doesn't do well, he will commit suicide. I don't know what I'm gonna do if I don't perform the way I should. But I certainly know that I will die if he does. In case everything goes wrong, I'll leave. There's no other way. The day of my death is nearing. It's just 3 months away. Love you all. Have a good day.
1172
| title | selftext | author | num_comments | is_suicide | url | |
|---|---|---|---|---|---|---|
| 0 | Our most-broken and least-understood rules is ... | We understand that most people who reply immed... | SQLwitch | 133 | 0 | https://www.reddit.com/r/depression/comments/d... |
| 1 | Regular Check-In Post | Welcome to /r/depression's check-in post - a p... | circinia | 1644 | 0 | https://www.reddit.com/r/depression/comments/e... |
| 2 | I hate it so much when you try and express you... | I've been feeling really depressed and lonely ... | TheNewKiller69 | 8 | 0 | https://www.reddit.com/r/depression/comments/f... |
| 3 | I went to the hospital because I was having re... | I literally broke down crying and asked to go ... | Jazzlecrab | 15 | 0 | https://www.reddit.com/r/depression/comments/f... |
| 4 | Cake day for me | Any kind soul want to give a depressed person ... | Depressed_Kid786 | 37 | 0 | https://www.reddit.com/r/depression/comments/f... |
| ... | ... | ... | ... | ... | ... | ... |
| 1892 | I'm in so much pain | It's so hard just to get out of bed every morn... | BigPete543 | 6 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 1893 | I'm too stupid to achieve anything in life. I'... | I have a low IQ. Twice I've been tested and bo... | Elisbt | 3 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 1894 | Useless | I don't feel useless, im useless. | Gayhova | 5 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 1895 | I wish it was as easy as disappearing into thi... | I wish there was some way I could will myself ... | evofe | 1 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 1896 | i dont know what to do with myself right now | i want to die but most of all i think i want t... | ponkichii | 3 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
1897 rows × 6 columns
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1897 entries, 0 to 1896 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 title 1897 non-null object 1 selftext 1832 non-null object 2 author 1897 non-null object 3 num_comments 1897 non-null int64 4 is_suicide 1897 non-null int64 5 url 1897 non-null object dtypes: int64(2), object(4) memory usage: 89.0+ KB
| title | selftext | author | num_comments | is_suicide | url | |
|---|---|---|---|---|---|---|
| 184 | Can someone recommend good qualities in therap... | NaN | eito_8 | 2 | 0 | https://www.reddit.com/r/depression/comments/f... |
| 921 | Nobody gives a fuck until you die, and even th... | NaN | lil_peemis | 3 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 923 | I have two brothers who have killed themselves... | NaN | ArsenalOwl | 1 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 925 | I want to die I want to die I want to die | NaN | alynde | 4 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 934 | I am so sorry, but it has gotten worse | NaN | SmushyKidK | 4 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 937 | I want to douse my body in gasoline and set my... | NaN | SalehRobbins | 3 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 941 | I can't do this anymore | NaN | sappy_banana | 4 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 957 | If I had a gun, I’d blow my fucking brains out... | NaN | CGM2004 | 1 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 960 | This world is a joke. | NaN | crybaby1577 | 11 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 964 | I think I’m ready | NaN | ___horse___ | 4 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
1 64 0 1 Name: is_suicide, dtype: int64
| title | selftext | author | num_comments | is_suicide | url | |
|---|---|---|---|---|---|---|
| 184 | Can someone recommend good qualities in therap... | emptypost | eito_8 | 2 | 0 | https://www.reddit.com/r/depression/comments/f... |
| 921 | Nobody gives a fuck until you die, and even th... | emptypost | lil_peemis | 3 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 923 | I have two brothers who have killed themselves... | emptypost | ArsenalOwl | 1 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 925 | I want to die I want to die I want to die | emptypost | alynde | 4 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
| 934 | I am so sorry, but it has gotten worse | emptypost | SmushyKidK | 4 | 1 | https://www.reddit.com/r/SuicideWatch/comments... |
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1897 entries, 0 to 1896 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 title 1897 non-null object 1 selftext 1897 non-null object 2 author 1897 non-null object 3 num_comments 1897 non-null int64 4 is_suicide 1897 non-null int64 5 url 1897 non-null object dtypes: int64(2), object(4) memory usage: 89.0+ KB
As the posts are written by different humans, they come in different forms. In order to prepare the data for our classifier, we will have to take steps to pre-process the posts.
Build processing functions - We will build a processing function that will help change the text to lowercase, remove punctuations, reduce related words down to a common base word. With this functions, we can create a seperate column for our clean data.
[nltk_data] Downloading package wordnet to [nltk_data] C:\Users\nisar\AppData\Roaming\nltk_data... [nltk_data] Package wordnet is already up-to-date! [nltk_data] Downloading package stopwords to [nltk_data] C:\Users\nisar\AppData\Roaming\nltk_data... [nltk_data] Package stopwords is already up-to-date!
| title | selftext | author | num_comments | is_suicide | url | selftext_clean | title_clean | |
|---|---|---|---|---|---|---|---|---|
| 0 | Our most-broken and least-understood rules is "helpers may not invite private contact as a first... | We understand that most people who reply immediately to an OP with an invitation to talk private... | SQLwitch | 133 | 0 | https://www.reddit.com/r/depression/comments/doqwow/our_mostbroken_and_leastunderstood_rules_is/ | understand people reply immediately op invitation talk privately mean help type response usually... | broken least understood rule helper may invite private contact first resort made new wiki explain |
| 1 | Regular Check-In Post | Welcome to /r/depression's check-in post - a place to take a moment and share what is going on a... | circinia | 1644 | 0 | https://www.reddit.com/r/depression/comments/exo6f1/regular_checkin_post/ | welcome r depression check post place take moment share going accomplishment want talk standalon... | regular check post |
| 2 | I hate it so much when you try and express your feelings to your parents, but they turn it aroun... | I've been feeling really depressed and lonely lately from my job, I'm a full time late night jan... | TheNewKiller69 | 8 | 0 | https://www.reddit.com/r/depression/comments/fedwbi/i_hate_it_so_much_when_you_try_and_express_y... | feeling really depressed lonely lately job full time late night janitor courthouse 10 mile away ... | hate much try express feeling parent turn around compare suffering |
| 3 | I went to the hospital because I was having really bad panic attacks, and theycontinued in there... | I literally broke down crying and asked to go home, another nurse was watching me while I was cr... | Jazzlecrab | 15 | 0 | https://www.reddit.com/r/depression/comments/feel0k/i_went_to_the_hospital_because_i_was_having/ | literally broke cry asked go home another nurse wa watching wa cry saying wanted keep longer kep... | went hospital wa really bad panic attack theycontinued ended collapsing nurse wa trying help sai... |
| 4 | Cake day for me | Any kind soul want to give a depressed person karma.\n\n\nEdit: Thanks everyone and especially f... | Depressed_Kid786 | 37 | 0 | https://www.reddit.com/r/depression/comments/fe6ua3/cake_day_for_me/ | kind soul want give depressed person karma edit thanks everyone especially sub hope overcome dep... | cake day |
| 5 | sorry but why the fuck is life like this | literally though. have i done anything wrong | angelshum10 | 13 | 0 | https://www.reddit.com/r/depression/comments/fecd9s/sorry_but_why_the_fuck_is_life_like_this/ | literally though done anything wrong | sorry fuck life like |
| 6 | I hate love | why does it hurt so much? Why can’t I be happy without it? There’s this empty void in my heart t... | DaRkNeSSReDCrusader | 29 | 0 | https://www.reddit.com/r/depression/comments/feb4tq/i_hate_love/ | doe hurt much happy without empty void heart get bigger everyday waiting eats since never 2 side... | hate love |
| 7 | This world has no place for people like me. | A bit of context. I'm an American. I'm black. I'm big. I'm ugly. And I'm autistic. I work as an ... | InsertANameHeree | 2 | 0 | https://www.reddit.com/r/depression/comments/fegd6t/this_world_has_no_place_for_people_like_me/ | bit context american black big ugly autistic work aircraft mechanic go every day le see point co... | world ha place people like |
| selftext | selftext_clean | |
|---|---|---|
| 0 | We understand that most people who reply immediately to an OP with an invitation to talk privately mean only to help, but this type of response usually leads to either disappointment or disaster. it usually works out quite differently here than when you say "PM me anytime" in a casual social context. \n\nWe have huge admiration and appreciation for the goodwill and good citizenship of so many of you who support others here and flag inappropriate content - even more so because we know that so many of you are struggling yourselves. We're hard at work behind the scenes on more information and resources to make it easier to give and get quality help here - this is just a small start. \n\nOur new wiki page explains in detail why it's much better to respond in public comments, at least until you've gotten to know someone. It will be maintained at /r/depression/wiki/private_contact, and the full text of the current version is below.\n\n*****\n\n###Summary###\n\n**Anyone who, while a... | understand people reply immediately op invitation talk privately mean help type response usually lead either disappointment disaster usually work quite differently say pm anytime casual social context huge admiration appreciation goodwill good citizenship many support others flag inappropriate content even know many struggling hard work behind scene information resource make easier give get quality help small start new wiki page explains detail much better respond public comment least gotten know someone maintained r depression wiki private_contact full text current version summary anyone acting helper invite accepts private contact e pm chat kind offsite communication early conversion showing either bad intention bad judgement either way unwise trust pm anytime seems like kind generous offer might perfectly well meaning unless solid rapport ha established wise idea point consider offer accept invitation communicate privately posting supportive reply publicly help people op respons... |
| 1 | Welcome to /r/depression's check-in post - a place to take a moment and share what is going on and how you are doing. If you have an accomplishment you want to talk about (these shouldn't be standalone posts in the sub as they violate the "role model" rule, but are welcome here), or are having a tough time but prefer not to make your own post, this is a place you can share.\n\nWe try our best to keep this space as safe and supportive as possible on reddit's wide-open anonymity-friendly platform. The community rules can be found in the sidebar, or under "Community Info" in the official mobile apps. If you aren't sure about a rule, please ask us. | welcome r depression check post place take moment share going accomplishment want talk standalone post sub violate role model rule welcome tough time prefer make post place share try best keep space safe supportive possible reddit wide open anonymity friendly platform community rule found sidebar community info official mobile apps sure rule please ask u |
| title | title_clean | |
|---|---|---|
| 0 | Our most-broken and least-understood rules is "helpers may not invite private contact as a first... | broken least understood rule helper may invite private contact first resort made new wiki explain |
| 1 | Regular Check-In Post | regular check post |
| 2 | I hate it so much when you try and express your feelings to your parents, but they turn it aroun... | hate much try express feeling parent turn around compare suffering |
| 3 | I went to the hospital because I was having really bad panic attacks, and theycontinued in there... | went hospital wa really bad panic attack theycontinued ended collapsing nurse wa trying help sai... |
| 4 | Cake day for me | cake day |
{'SQLwitch': 'SQL witch', 'circinia': 'c irc in i a', 'TheNewKiller69': 'The New Killer 69', 'Jazzlecrab': 'Jazz le crab', 'Depressed_Kid786': 'Depressed Kid 786', 'angelshum10': 'angels hum 10', 'DaRkNeSSReDCrusader': 'DaRkNeSS ReD Crusader', 'InsertANameHeree': 'Insert A Name Here e', 'chroms-leather-belt': 'ch roms leather belt', 'deppressionthrowaway': 'dep press ion throwaway'}
| author | author_clean | |
|---|---|---|
| 1887 | michael5976 | michael 5976 |
| 1888 | sprinklesbubbles123 | sprinkle bubble 123 |
| 1889 | Francosviky | franco vi ky |
| 1890 | SeaSaltSaw | sea salt saw |
| 1891 | crybaby1577 | crybaby 1577 |
| 1892 | BigPete543 | big pete 543 |
| 1893 | Elisbt | eli sbt |
| 1894 | Gayhova | gay h ovum |
| 1895 | evofe | ev e |
| 1896 | ponkichii | pon k ichi |
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1897 entries, 0 to 1896 Data columns (total 9 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 title 1897 non-null object 1 selftext 1897 non-null object 2 author 1897 non-null object 3 num_comments 1897 non-null int64 4 is_suicide 1897 non-null int64 5 url 1897 non-null object 6 selftext_clean 1897 non-null object 7 title_clean 1897 non-null object 8 author_clean 1897 non-null object dtypes: int64(2), object(7) memory usage: 133.5+ KB
title 0 selftext 0 author 0 num_comments 0 is_suicide 0 url 0 selftext_clean 0 title_clean 0 author_clean 0 dtype: int64
We now have three possible columns to build our classifier on: "author_clean", "selftext_clean", "title_clean". Now, on to EDA.
Some areas to check out:
Top words - This would be our obvious first step in our EDA. To peek and see what are the most used words in the title, posts and usernames.
Significant Authors - This might not really affect the classifier that we are using, but it might be worth it to check out users who post often and users who has posted on both subreddits.
Average length of posts - We have already noticed a significant number of posts with no words in r/SuicideWatch posts. We should dive in to check out what the number of words are in an average post in each subreddit.
--------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-3-6926f8a89ed0> in <module> 1 #USING MASKS TO CREATE SLICES OF OUR DATAFRAME FOR VISUALISATION ----> 2 suicide_posts = combined_data[combined_data["is_suicide"] ==1]["selftext_clean"] 3 suicide_titles = combined_data[combined_data["is_suicide"] ==1]["title_clean"] 4 suicide_authors = combined_data[combined_data["is_suicide"] ==1]["author_clean"] 5 NameError: name 'combined_data' is not defined
We will visualise the most-used words in the respective subreddit posts in a word cloud and a barplot. We will first define a function that will help us with that. The function can be re-used for our titles and usernames later.
--------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-2-777606084132> in <module> 2 #WE'LL USE DIFFERENT PALETTES TO SEPERATE THE TWO REDDITS: 3 #THE PENSIVE "ocean_r" FOR r/depression AND THE HEAVY AND DARK SHADES OF "magma" FOR FOR r/SuicideWatch ----> 4 plot_most_used_words("r/depression Posts", depression_posts, palette="ocean_r", image_mask="../assets/depression_mask.png") NameError: name 'depression_posts' is not defined
Many similar words - We see a massive amount of similar words in our "top 20 words" from both subreddits like: wa, want, like, feel, life and people. This might make it difficult four our model.
Unique words from r/depression - "depression"
Unique words from r/SuicideWatch - "anymore"
We will visualise the most-used words the our respective subreddit titles in a word cloud and a barplot.
Titles a better differentiator? - We still see a fair amount of similar words in our "top 20 words" in our titles, but much less than in Posts. As Titles in the two subreddits seem to differ more than Posts, it might serve as a better place for our models to hunt for features.
Unique words from r/depression titles - "depression", "depressed", "hate"
Unique words from r/SuicideWatch titles - "die", "kill", "suicidal", "live", "year"
The difference between Wanting and Feeling - It is interesting to note that the top word in r/SuicideWatch is the word "want" and it is used more than twice compared to the word "feel". In r/depression, the top word is "feel" and similarly, is used close to twice the amount of times compared to "want". This trend is also reflected in our visualisations for posts.
We will visualise our most-used words in the respective subreddit usernames in a word cloud and a barplot. The choice of exploring usernames is an odd one as usernames are really short and it'll be really surprising if we can find something revealing in a sample pool of 1897 usernames. Let's try any way!
67
39
29
68
15
1051 bay girl ad girl 1508 throwaway girl 83 1542 sad girl throwaway 6 1619 disposable baby girl 1836 crazy smile girl Name: author_clean, dtype: object
323
349
14 toasty 420 162 daniel quer 420 236 cereal 420 bla zi ken 302 taco cat 6942042 361 hyper aid 420 366 der bag 420 644 th icc barbie 420 772 su kari 420 811 mom destroyer 69420 1231 star rider 420 1277 penis wrinkle 69420 1388 th icc barbie 420 1433 spa main 420 1482 wiener schnitzel z 420 1710 fuck boi 420 69 Name: author_clean, dtype: object
1897
Throwaways Dominate - There are 67 accounts(out of 1897) with the word "throwaway" in it. Throwaway accounts are temporary accounts used by users who want to maintain some anonymity. This is understandable given the subject matter of mental health.
Male-signifiers - Our top 20 list for depression is dominated by usernames with "mr", "man", "boy", "guy" in them. A check in on our full list of authornames revealed that there are more male-related names(68) than female ones(15). Its not a strong link our finding, but it is worth noting that there is a gender paradox in suicide studies, which observes the phenomenon of women having more suicidal thoughts while men commit suicide more frequently.
Marijuana - The 420, or "four-twenty", cannabis code made its way into our top 20 for depression. It might be worthwhile to look for links to drug-use in our posts.
Usefulness of Usernames? - Although our discoveries in Usernames are interesting, the "Top Words" occur in an average of 5 usernames. This might give our model some trouble. "Title" is still my top choice as the column to use to search for features.
As an attempt to understand the community in the pages, we will attempt check out users who post often and users who has posted on both subreddits.
[deleted] 11 Vivid-Smile 5 snakesnack148 4 outakuslayer69 4 throaway8297338 4 unicorn6642 3 enk9898 3 LifeisCrumbling 3 SeaSaltSaw 3 SuperDuperPewper 3 lIlFlashIlI 3 missmisery8 3 AlinaAngelMtF 3 vadimuha 3 TheGrandMan10 3 SQLwitch 3 SilentPost3 3 tiredlucifer 2 floorecent-stare 2 wasps_in_my_anus 2 Name: author, dtype: int64
| is_suicide | author | title | selftext | |
|---|---|---|---|---|
| 0 | 0 | SQLwitch | Our most-broken and least-understood rules is "helpers may not invite private contact as a first resort", so we've made a new wiki to explain it | We understand that most people who reply immediately to an OP with an invitation to talk privately mean only to help, but this type of response usually leads to either disappointment or disaster. it usually works out quite differently here than when you say "PM me anytime" in a casual social context. \n\nWe have huge admiration and appreciation for the goodwill and good citizenship of so many of you who support others here and flag inappropriate content - even more so because we know that so many of you are struggling yourselves. We're hard at work behind the scenes on more information and resources to make it easier to give and get quality help here - this is just a small start. \n\nOur new wiki page explains in detail why it's much better to respond in public comments, at least until you've gotten to know someone. It will be maintained at /r/depression/wiki/private_contact, and the full text of the current version is below.\n\n*****\n\n###Summary###\n\n**Anyone who, while a... |
| 918 | 1 | SQLwitch | Reminder: Absolutely no activism of any kind is allowed here. Any day. | If you want to recognise an occasion, please do so by offering extra support to those who've asked for it, and by being a good citizen of the community, mindful of our tips and guidelines.\n\nEdit: In response to multiple requests, I'm adding a definition of activism here: "campaigning to bring about political or social change". That's straight out of the OED, we aren't making any special interpretation of it. **Any post that is primarily about raising awareness or changing perceptions is activism according to the dictionary definition and thus a violation of the community rules.**\n\nSome examples of the most common types of activists posts we were here, all of which are totally inappropriate: \n\n* Advocating for changes in laws around suicide\n\n* Advocating for change in the mental-health system \n\n* Advocating for changes in cultural norms or common perceptions around mental-health. These are often related to specific days devoted to specific causes, e.g. World Suicide P... |
| 917 | 1 | SQLwitch | New wiki on how to avoid accidentally encouraging suicide, and how to spot covert incitement | We've been seeing a worrying increase in pro-suicide content showing up here and, and also going unreported. This undermines our purpose here, so we wanted to highlight and clarify our guidelines about both direct and indirect incitement of suicide. \n\nWe've created a wiki that covers these issues. We hope this will be helpful to anyone who's wondering whether something's okay here and which responses to report. It explains in detail why *any* validation of suicidal intent, even an "innocent" message like "if you're 100% committed, I'll just wish you peace" is likely to increase people's pain, and why it's important to report even subtle pro-suicide comments. The full text of the wiki's current version is below, and it is maintained at [/r/SuicideWatch/wiki/incitement](http://www.reddit.com/r/SuicideWatch/wiki/incitement). \n\nWe deeply appreciate everyone who gives responsive, empathetic, non-judgemental support to our OPs, and we particularly thank everyone who's already been... |
| 1515 | 1 | Vivid-Smile | Can't forget the girl I love since high school | I can't forget her. I love her. I want her so much \n\nBut she hates me. Blocked for a year.\n\nI'm 25 and never had a relationship \n\n\nI feel humiliated that I never got a proper chance \n\nIf I can't be with her, I will kill myself |
| 1241 | 1 | Vivid-Smile | I'm going to end my life over a girl | That's it.\n\n\nI can't stand it anymore. \n\nIt's my family's fault and they refuse to keep promises |
| 1218 | 1 | Vivid-Smile | That's it. I'm going to kill myself tomorrow | My "family" destroyed my life\n\n\nI'll never see the girl I like because of them and refuse to do anything about it.\n\nKeep blaming me for everything.\n\n\nI'm done |
| 991 | 1 | Vivid-Smile | I have a belt around my neck. I wish I knew how to do it properly | emptypost |
| 995 | 1 | Vivid-Smile | Things are worse. I gave it a year and nothing changed | emptypost |
| 1100 | 1 | enk9898 | I got the gun | It’s over I can’t believe it I never thought I’d get this. driving to a quiet park and I’m gonna call the cops before I do it. So I won’t be to smelly to come clean it up. I’m lining the car with garbage bags. I’m sorry for anyone that has to suffer here on this earth in your body when it is the thing you resent most. I hope you find peace I will soon. Maybe you’ll hear about me in the news I’m from PA not much information. But just a hint I guess. \n\nThank you good luck and goodbye see ya on the other side. |
| 1334 | 1 | enk9898 | Picking up a gun tomorrow | I’m relieved |
| 1419 | 1 | enk9898 | Officially homeless! | Mom kicked me out:) can’t wait for this to be the final straw to kill myself I wish I was never 302 so I could get a gun :/ I’ll have to sleep in the car a few nights till I get my hands on one. |
| 632 | 0 | outakuslayer69 | clap, clap meme review! | depression is real \n\n&#x200B;\n\n&#x200B;\n\n&#x200B;\n\nplaylist: [https://open.spotify.com/playlist/2Yu2ynndFtdS8ki6YjLmDu?si=uaZDtmVsSVm8FjJs3W-PyQ](https://open.spotify.com/playlist/2Yu2ynndFtdS8ki6YjLmDu?si=uaZDtmVsSVm8FjJs3W-PyQ) |
| 168 | 0 | outakuslayer69 | how to deal with depression | how to deal with depression \n\n1 give me an upvote maybe \n\n2 [https://open.spotify.com/playlist/2Yu2ynndFtdS8ki6YjLmDu?si=fqkBX4P8TTG1fcapz-FFyw](https://open.spotify.com/playlist/2Yu2ynndFtdS8ki6YjLmDu?si=fqkBX4P8TTG1fcapz-FFyw) |
| 755 | 0 | outakuslayer69 | hope l die soon hahahahahha | don´t agree \n\n&#x200B;\n\nSpotify account[https://open.spotify.com/user/iy60tufjnk1h2uufwng66lb4e?si=fx7lKbdMSrC6Omrh7myTtA](https://open.spotify.com/user/iy60tufjnk1h2uufwng66lb4e?si=fx7lKbdMSrC6Omrh7myTtA) \n\nplaylist [https://open.spotify.com/playlist/2Yu2ynndFtdS8ki6YjLmDu?si=USIoiZo8QdSX56k1zg\_lwQ](https://open.spotify.com/playlist/2Yu2ynndFtdS8ki6YjLmDu?si=USIoiZo8QdSX56k1zg_lwQ) |
| 1884 | 1 | outakuslayer69 | yesterday I died in my dream, what a wonderful dream that was. | **depressed people wanna live too.... you know hahaha!**\n\n**(kill me pls)**\n\n[**https://open.spotify.com/user/iy60tufjnk1h2uufwng66lb4e?si=t\_1QVucLR6SiEKCf4qyCiQ**](https://open.spotify.com/user/iy60tufjnk1h2uufwng66lb4e?si=t_1QVucLR6SiEKCf4qyCiQ)\n\n**l got really depressing song on my Spotify playlist if you wanna hear?** |
| 1854 | 1 | snakesnack148 | What happened to you when you overdosed and how much did you take? | Since so much people have overdosed on this subreddit I’m curious |
| 1369 | 1 | snakesnack148 | I feel so low it physically hurts | My heart hurts from the pain, I wish I could just disappear, my parents have gone through so much because of me, they do so much to make me happy ,Even my siblings are jealous, I feel so guilty they have to deal with me, see their child try to kill themselves everyday, crying from depression. I just want them to know I love them so much, I can’t even describe how much I love them, and I’m so sorry for what I’m doing, what I am feeling, what I am going to do in the future, I would love to change but I can’t.\n\nI just wish there is something I can do to feel happy again. |
| 1492 | 1 | snakesnack148 | Fastest way to become ill? Enough to get hospitalized | So I have to go to a psychiatric ward soon, and I don’t want to go back so what Is the fastest way to become ill? Enough to stay in general hospital? \nI really need help so please reply as soon as possible this is my only option. |
| 1518 | 1 | snakesnack148 | Need your help | I go to this sub when ever I need help, and today I really need help, how do I make myself ill? I really need to get ill so I don’t go back to the psychiatric ward but instead go to general hospital, please help. |
| 638 | 0 | throaway8297338 | Don’t know how to keep going with the award system broken in my brain | I see the brick walls where I used to see beauty. Eating no longer feels pleasurable. My hobbies make me agitated. My friends make me anxious.\nThe only part of my brain that still works is the fear response. \n\nI’m not sure how much longer I can keep going. I grew up being taught to do the right thing—and I know continuing on is the right thing—but I didn’t realize how much human sanity relied on positive emotions. We are not guided by instinct—we are guided by these emotions. Mine are directing me towards death, and I wonder why, in the most advanced on minds, this pathway exists (suicide is unique to humans even though animals kill themselves). \n\nYou know that feeling when you get your heartbroken? I feel that everyday, but I didn’t lose someone. I lost everyone and everything. I lost life. |
| 455 | 0 | throaway8297338 | There is no rock bottom—just an endless abyss with ledges | It can always get better but it can always get worse too. |
| 204 | 0 | throaway8297338 | Am I weak or do I not have a choice? | Why can’t I do what I need to do? I remember telling my final therapist how I’d sit paralyzed while my mind screamed at me to do things, and she said I just had to do them. \n\nIt was one of those things I’ve been told that stings to this very day. Is this paralysis some internal weakness that I do have control over? If so, how the hell does this work because I can’t figure it out?\n\nI wouldn’t have gone to see someone if I’d have known they’d told me what they did. I came to them for help. Was what they gave me the only help there is? Is this my fault? Or do I even have a choice being like this? |
| 56 | 0 | throaway8297338 | I want to die | This is not a passing or intrusive thought. It’s there even when I feel OK. \n\nI’m ready to go now. |
| is_suicide | author | title | selftext | |
|---|---|---|---|---|
| 278 | 0 | [deleted] | I'm tired of the pain. | I just can not keep doing this anymore. I keep reaching out for help, doctors push pills on me (I'm on five now) and therapists just try to push me into blaming my dead father. \n\nI can't afford my home. I can't afford my bills. I can't afford food and even the subreddits that claim to offer help just delete my posts because I lurk and my account doesn't have hundreds of karma.\n\nI do not know what to do. I cry constantly. I hate my life and myself. I can't sleep I can't eat. \n\nI try to open up to people and they look at me like a monster.\n\nI am a monster.\n\nI'm about to lose my home and I have nothing left to sell to try to hang on to it. I sold all my dads belongings to make ends meet, he left those to me so I could have a good fun life and here I am a piece of garbage who can't even work a part time job to buy groceries. \n\nThe state turns me away because I have income ($944/mo in disability).\n\nI'm done. I can't do this. Why am I fighting? For who? I'm alone. My chest ... |
| 329 | 0 | [deleted] | I’ve lost 4 pounds since Monday, this is just getting worse | I don’t like to complain and so I don’t talk about my depression a lot to my friends/loved ones, I don’t want to worry them or be annoying. It’s been getting progressively worse recently though, I’m dropping weight like crazy (I’ve never had it interfere with weight before), when I do try to talk about it, things get minimized and nobody really gives a f*ck. I had my s/o tell me that I was “not really sad” yesterday, and that messed with me because not only has he not known me for the majority of my life, but just because I don’t like to talk about my problems doesn’t mean they don’t exist. He really has no idea. My mom is a therapist so when I try to talk to her she just gets overly worried, and my dad generally follows her lead with that. I understand why they worry, a couple years ago I had a pretty bad suicide attempt and a near fatal overdose (od was an accident, I was just being dumb) within less than 6 months of each other, they worry that it’ll happen again. It gets annoyin... |
| 417 | 0 | [deleted] | I'm tired of all the noise | I think I get misinterpreted a lot. I don't feel good. I'm selfish but I need validation too. My things matter but so do yours. I just need to find the right meds. |
| 498 | 0 | [deleted] | Totally burned out and extrem depressed | So im going to tell you a „little“ about my Situation. I am usually a pretty happy Person. I like doing fun Things like going out with Friends who ive known pretty much all my life (im 23 ) and i have a passion which Is Film making. But Since i started with school even at a Young age i always had Problems concentrating and focusing at School it just isnt for me because I don’t see the benefits from it. I also think the school system is extremly outdated. Last September I started my education ( I live in Germany and the English word for ausbildung translates to education.. ) I work in a world known firm which im not going to name because of a couple of reasons which aren’t related to my problem though. I would say it’s pretty fun because all of my co workers and bosses are pretty nice and always want me to succeed and do good. I’m also pretty good at this job. Even though I got a lot for me going in this particular job I don’t really see me self working in this job for the rest of m... |
| 512 | 0 | [deleted] | I’m losing myself. | I can’t keep going.\n\nI try and try to be a good person and every time I turn around the world takes something from me. I can’t afford to live anymore. The state of the world makes me sick. My family is so splintered. I lost my dad and my mother is so beyond done dealing with my messed up self it makes my chest hurt.\n\nI never wanted to be this. I wanted to make my family’s life better but here I am a failed 38 year old man who can’t even support himself let alone help those I love.\n\nI don’t know where to turn. Doctors push so many drugs on me and I feel like I can not even think because of it therapy just wants me to blame my father and I do t want to do that, I loved him and he loved me. \n\nI hate myself and who I have become. I want a normal life. I want to support myself and help my loved ones. I was the rock and now I’m just a pile of shit.\n\nI can’t do this. I hate it, I hate me. I do not remember a time when my chest did not hurt from this. I’m so tired. I want to love... |
| 591 | 0 | [deleted] | Red Dead Redemption 2 helps a lot. | I've been getting progressively worse throughout this year, and I found myself having the exact same days over and over again. My social life is pretty much nonexistent at this point, and I've found myself struggling to hold onto friends that are drifting away from me.\n\nThen I started playing this game, and I'm actually excited to get back to my apartment and spend time in this world. It feels so rewarding even just to spend the whole day hunting or working on a level 10 beard. It gives me a sense of purpose I haven't really been feeling lately. I'm probably going to spend about a month in each chapter of the game because I want to prolong my time playing.\n\nJust wanted to share my experiences as this game has definitely made my life more positive in the last month. |
| 746 | 0 | [deleted] | see mom, it's not a phrase! | I'm a grown ass adult and still want to fucking kill myself every single day! |
| 1288 | 1 | [deleted] | This is my first and last Reddit post | ...because I’m killing myself in the next few hours. I wanted to be able to push through all the pain that I’ve been in for so long and I wanted to believe that it could get better for me, but the waiting isn’t worth it. It’s not getting better and I’m too weak to keep dealing with this day in and day out. I’m sorry that this will hurt people who knew me — probably people who I don’t even think care about me. But I need to be selfish for today and do something for myself and put an end to this. \n\nGoodbye. |
| 1549 | 1 | [deleted] | Called off my work , going to end it after I eat | Nice knowing you all\n\nI'm finally coming home Mara and Tara ❤️ |
| 1675 | 1 | [deleted] | By next week | I'll be using a certain method that is inhaling and when those two chemicals are mixed together , it has a 100 % fatality rate. After next Monday when I grab something to eat and a beer then I'll go to a place in the hills or somewhere where I won't be stopped and complete the method. Soon I'm going to be asleep and forever away from human suffering and be free if any issues for eternity ❤️ |
| 1708 | 1 | [deleted] | I dont want to die. I dont want to keep breathing. | I use to drink often, I stopped. I'm very intoxicated so I might be all over. Shit happens right. \n\nI've been depressed for so fucking long it hurts to think about. At 13, 16, 19 everyone says it'll get better. I'm 23 and nothing's gotten any better.\n\nI had a 3 year relationship and she threw me to the curb. I had a 2 year relationship following (after a break from the 1st, we only dated a year but were close/fuck buddies for a year leading to it) and we broke up tonight, fun fact we still live together until july. My depression doesn't stem from relationships, there's a plethora of things contributing to it. I think I'm out of my depth on this one. We were fuck buddies because I didn't want another relationship. I didn't want to give someone a chance to hurt me again. \n\nShe was moving for school and I couldn't bare the thought of losing her so I asked her to date and we planned and did move in together.\nI've punched myself in the face because it's less damaging then cuttin... |
['[deleted]', 'Vivid-Smile', 'snakesnack148', 'outakuslayer69', 'throaway8297338', 'unicorn6642', 'enk9898', 'LifeisCrumbling', 'SeaSaltSaw', 'SuperDuperPewper', 'lIlFlashIlI', 'missmisery8', 'AlinaAngelMtF', 'vadimuha', 'TheGrandMan10', 'SQLwitch', 'SilentPost3', 'tiredlucifer', 'floorecent-stare', 'wasps_in_my_anus', 'anxxxer', 'lovestar777111', 'AbsorbedSky312', 'YayWereAllGonnaDie', 'iwannadiebadly', 'RegularlyAlive', 'Jimmyconwayyy', 'Jackbright682', 'thathumbletrashcan', '_dojibe_kara', 'Boogiedat', 'PupCorn_', 'AnotherTrowaway12', 'zdento', 'deppressionthrowaway', 'Tyler080721', 'SmarmySlayer', 'OutsideitCZ', 'runningrunningrunrun', 'BamaSOH', 'Jefvv', 'gutsthrowaway', 'Timoris', 'peee-poopy-pee-head', 'roadtonowhere5', 'DailyDoseDepression', 'zamas2', 'Aimsm', 'cheesiemilk', 'iwaki-rogan', 'throwaway656565167', 'throwawaygoaway30', 'crashboxer1678', 'SandalFishing', 'rodbsbsb', 'd3ad1ns1de', 'thebus01', 'Souless_F0X', 'walker310', 'TroyBehave', 'zn07', 'rememberthememoriez', 'gr33nthundah', 'abadassbitch_2000', 'MrMoonlight4', 'brownted123', '17yrcub', 'bunnybelljpeg', 'gafsr', 'DrakenJosh98', '1sadbish', 'anxietyqueen101', 'Hlp4me', 'thiccbarbie420', 'Senseless_misery', 'ShocktasticAnimation', 'log1992', 'CovertGamesINC', 'idontwannagonow', 'angelasyy', 'chillinlookin', 'Saikujo', 'TheHours327', 'chivvg', '1lbOfViettiBeefStew', 'MajiKoju', 'catattack003', 'theCommonSlaw', 'southwestperson', 'okcomptr', 'puriuh', 'Iamoriginalthrowaw4y', 'pekelniydrochun', 'Isabella12345678910', 'SpookDaddy-', 'beauTeeEdur', 'The_Answer_Is_42__', 'anonanonnobidy', 'grande__oso', 'libertysailor', 'gettingbetter95', 'ajlaba', 'dereulb', 'LifeIsHorrible002', 'lostluos', 'findingstrengthtokms', 'justguylol', 'goodNonEvilHarry', 'Smurphy55656', 'marsupial65', 'AeonianAce', 'nidham21', 'smolmight-_-', 'Jeeseeka', '1hsayg1', 'yathatscooliguess', 'ScaryMohawk', 'Talking_About_Stuff', 'imjustbadbadnews', 'mjdg25', 'R78n', 'Sinigamic1', 'osir1s2', 'Hinatasamaa', 'MiaNebulosaX', 'ovrprotectiveunicorn', 'honeybee_hive', 'helpjustcut', 'MichiDraws', 'cockersPAINel', 'cassie7189', 'Throwaway32414111111', 'Joanh9000', 'buhrainbow', 'crybaby1577', 'NussNougatCroissant', 'annifrank1723', 'roybttynexvs6', 'oeyo121', 'veelw', 'elianystinks', 'Celestialsmoothie28', 'the-greatest-paninis', 'birbeech']
27
<ipython-input-49-b7033dcdca91>:11: UserWarning: Boolean Series key will be reindexed to match DataFrame index.
double_posters = more_than_once_mean_df[double_posters_mask_0][double_posters_mask_1].sort_values("num_comments", ascending=False)
['SQLwitch', 'buhrainbow', 'dereulb', 'RegularlyAlive', 'deppressionthrowaway', 'thathumbletrashcan', 'vadimuha']
| is_suicide | author | title | selftext | url | |
|---|---|---|---|---|---|
| 135 | 0 | RegularlyAlive | Pretty sure my life is falling apart. | I'm a fucking burden on everyone. Idk why there's still like 5 people that care. I wish they didn't so I could just fucking die already | https://www.reddit.com/r/depression/comments/fegjav/pretty_sure_my_life_is_falling_apart/ |
| 1257 | 1 | RegularlyAlive | Does anyone else feel destined to die by suicide? | It sounds emo and cringy as hell, but I've felt this way for 8 years. Meh | https://www.reddit.com/r/SuicideWatch/comments/fd8tmk/does_anyone_else_feel_destined_to_die_by_suicide/ |
| 0 | 0 | SQLwitch | Our most-broken and least-understood rules is "helpers may not invite private contact as a first resort", so we've made a new wiki to explain it | We understand that most people who reply immediately to an OP with an invitation to talk privately mean only to help, but this type of response usually leads to either disappointment or disaster. it usually works out quite differently here than when you say "PM me anytime" in a casual social context. \n\nWe have huge admiration and appreciation for the goodwill and good citizenship of so many of you who support others here and flag inappropriate content - even more so because we know that so many of you are struggling yourselves. We're hard at work behind the scenes on more information and resources to make it easier to give and get quality help here - this is just a small start. \n\nOur new wiki page explains in detail why it's much better to respond in public comments, at least until you've gotten to know someone. It will be maintained at /r/depression/wiki/private_contact, and the full text of the current version is below.\n\n*****\n\n###Summary###\n\n**Anyone who, while a... | https://www.reddit.com/r/depression/comments/doqwow/our_mostbroken_and_leastunderstood_rules_is/ |
| 917 | 1 | SQLwitch | New wiki on how to avoid accidentally encouraging suicide, and how to spot covert incitement | We've been seeing a worrying increase in pro-suicide content showing up here and, and also going unreported. This undermines our purpose here, so we wanted to highlight and clarify our guidelines about both direct and indirect incitement of suicide. \n\nWe've created a wiki that covers these issues. We hope this will be helpful to anyone who's wondering whether something's okay here and which responses to report. It explains in detail why *any* validation of suicidal intent, even an "innocent" message like "if you're 100% committed, I'll just wish you peace" is likely to increase people's pain, and why it's important to report even subtle pro-suicide comments. The full text of the wiki's current version is below, and it is maintained at [/r/SuicideWatch/wiki/incitement](http://www.reddit.com/r/SuicideWatch/wiki/incitement). \n\nWe deeply appreciate everyone who gives responsive, empathetic, non-judgemental support to our OPs, and we particularly thank everyone who's already been... | https://www.reddit.com/r/SuicideWatch/comments/cz6nfd/new_wiki_on_how_to_avoid_accidentally_encouraging/ |
| 918 | 1 | SQLwitch | Reminder: Absolutely no activism of any kind is allowed here. Any day. | If you want to recognise an occasion, please do so by offering extra support to those who've asked for it, and by being a good citizen of the community, mindful of our tips and guidelines.\n\nEdit: In response to multiple requests, I'm adding a definition of activism here: "campaigning to bring about political or social change". That's straight out of the OED, we aren't making any special interpretation of it. **Any post that is primarily about raising awareness or changing perceptions is activism according to the dictionary definition and thus a violation of the community rules.**\n\nSome examples of the most common types of activists posts we were here, all of which are totally inappropriate: \n\n* Advocating for changes in laws around suicide\n\n* Advocating for change in the mental-health system \n\n* Advocating for changes in cultural norms or common perceptions around mental-health. These are often related to specific days devoted to specific causes, e.g. World Suicide P... | https://www.reddit.com/r/SuicideWatch/comments/d2370x/reminder_absolutely_no_activism_of_any_kind_is/ |
| 37 | 0 | buhrainbow | Being ugly just isn't fair | I had no choice to be e beautiful guy. Wherever I go I'm the ugliest person. I want a girlfriend, but I can't talk to girls, since they obviously don't wanna talk to an ugly guy. Damn, it is not fair. I'm a fun guy, I'm even a comedian, but all the girls that aproaches me becomes my friend. Nobody wanna date an ugly guy. Thank you, "God". | https://www.reddit.com/r/depression/comments/fduopy/being_ugly_just_isnt_fair/ |
| 1216 | 1 | buhrainbow | I'm... thinking... bad things | I'm tired. I can't see a purpose to still here. My mom says I was a mistake and she wanted to abort me. I grew up as a mistake. She once put out the cigarette on my knee. People say I'm a bad guy since ever. I'm ugly. I'm lonely. My job is awesome but I'm not worthy of it, as I spend my whole time procrastinating. I'm a fail, I'm a big mess. | https://www.reddit.com/r/SuicideWatch/comments/fduzrc/im_thinking_bad_things/ |
| 9 | 0 | deppressionthrowaway | 8 years ago, I posted here, wanting to die. My life is so much better now. | When I was 15-16 years old, I posted here in my darkest moments. I couldn't see any reason to keep living. I thought I was nothing special and the next 8 years seemed like an endless slog. You can look through my post history if you want. \n\nSince then, I graduated college and graduate school. I have traveled the world. I swam with bioluminescent algae in Indonesia. I've hiked among alpacas in Peru. I learned to speak Spanish fluently and connected with people around the world and in my own country, who speak Spanish too. I've published my writing i multiple publications. I have fallen in love and had my heart broken several times. I realized I'm gay, came out, and I'm with the most amazing woman. I have a career that I love where I help people every day. I can honestly say that depression may be a lifelong struggle for me, but it is possible to recover and develop tools to cope with it. Every day is a manageable challenge now. Every day I am grateful to just be alive. I am SO GL... | https://www.reddit.com/r/depression/comments/fe6pvs/8_years_ago_i_posted_here_wanting_to_die_my_life/ |
| 924 | 1 | deppressionthrowaway | 8 years ago I posted here, wanting to die. My life is so much better now. | When I was 15-16 years old, I posted here in my darkest moments. I couldn't see any reason to keep living. I thought I was nothing special and the next 8 years seemed like an endless slog. You can look through my post history if you want. \n\nSince then, I graduated college and graduate school. I have traveled the world. I swam with bioluminescent algae in Indonesia. I've hiked among alpacas in Peru. I learned to speak Spanish fluently and connected with people around the world, and in my own country, who speak Spanish too. I've published my writing i multiple publications. I have fallen in love and had my heart broken several times. I realized I'm gay, came out, and I'm with the most amazing woman. I have a career that I love where I help people every day. I can honestly say that depression may be a lifelong struggle for me, but it is possible to recover and develop tools to cope with it. Every day is a manageable challenge now. Every day I am grateful to just be alive. I am SO G... | https://www.reddit.com/r/SuicideWatch/comments/fe6pma/8_years_ago_i_posted_here_wanting_to_die_my_life/ |
| 348 | 0 | dereulb | Why are you still alive? | If you’re reading this, chances are you’re still alive. Why haven’t you killed yourself yet?\n\nSince I’m still alive, there must be a reason but all signs point to it not being worth it for me. Hopefully your reasons can help other people find theirs. | https://www.reddit.com/r/depression/comments/fe0ofw/why_are_you_still_alive/ |
| 1063 | 1 | dereulb | Why are you still alive? | If you’re reading this, then chances are you’re alive. Why haven’t you killed yourself yet?\n\nI’m trying to find my reason. | https://www.reddit.com/r/SuicideWatch/comments/fe0ngb/why_are_you_still_alive/ |
| 646 | 0 | thathumbletrashcan | I don't want to die, but I don't want to live anymore | I feel like people don't understand the fact that being suicidal is not all about "wanting to die". I don't want to die, I just want the pain I'm causing to myself and those around me by being a burden to end and dying seems like the most relevant solution as most of the other stuff I have tried has failed. I think that's why I oversleep a lot, that way I'm not really living, its like temporary death, death without commitment. I don't even know what I'm talking about I guess I just want the pain to end. | https://www.reddit.com/r/depression/comments/fdcqxp/i_dont_want_to_die_but_i_dont_want_to_live_anymore/ |
| 1535 | 1 | thathumbletrashcan | I've finally given in | This is it I guess, I've finally grown the balls to fulfil my plan. Hopefully, if all goes well and all of you won't have to deal with me again. Thanks for putting up with me for this long, I love yall. ✌️ | https://www.reddit.com/r/SuicideWatch/comments/fdgpf2/ive_finally_given_in/ |
| 603 | 0 | vadimuha | Has it ever become better? | I've been in a therapy for about 2 years now and it seems like I just got worse | https://www.reddit.com/r/depression/comments/fe05bv/has_it_ever_become_better/ |
| 1570 | 1 | vadimuha | I want to quit therapy and just make myself worse enough where I can finally kill myself | emptypost | https://www.reddit.com/r/SuicideWatch/comments/fd69wa/i_want_to_quit_therapy_and_just_make_myself_worse/ |
| 1659 | 1 | vadimuha | Why do people think they can help when you already desided on ending it all? | emptypost | https://www.reddit.com/r/SuicideWatch/comments/fd603h/why_do_people_think_they_can_help_when_you/ |
The Moderator - The redditor SQLWitch seems to be a moderating presence in both subreddits. Appearing a couple of times to remind redditors about rules like non-activism and not posting pro-suicide posts. SQLWitch's posts hints at the culture both communities, which are similar help-seeking, problem-airing forums.
Double-Posting - Aside from SQLWitch, there are more than 26 other users who have posted on both forums. For example, u/thathumbletrashcan posted on r/depression on March 4th that "I don't want to die, but I don't want to live anymore" . A day later, u/thathumbletrashcan visits the r/SuicideWatch forum and posts "I've finally given in. This is it I guess, I've finally grown the balls to fulfil my plan......all of you won't have to deal with me again." It seems that users treat r/SuicideWatch as a "late-stage" forum or a place to bid farewell to their fellow help-seekers. It might be useful to dig into the language used by these users prior to their "farewells" as the "shift" in their choice of words is exactly what we're looking for.
We have already noticed a significant number of posts with no words in r/SuicideWatch posts. We should dive in to check out what the number of words are in an average post in each subreddit.
Average length of a r/depression title: 39.31952017448201 Average length of a r/SuicideWatch title: 39.72244897959184 Average length of a r/depression post: 964.3217011995638 Average length of a r/SuicideWatch post: 836.3602040816327
Longer r/depression posts - The average length of r/depression posts is almost 130 words shorter than that of r/SuicideWatch. Although, as we can see from our scatterplot above, this figures might be skewed by some extremely long posts and the presence of empty posts in r/SuicideWatch.
As a final step in our EDA, we will use Scattertext to produce a user-friendly way of visualising our corpus in HTML.
| megatext_clean | is_suicide | category | |
|---|---|---|---|
| 1892 | big pete 543 hard get bed every morning right want die damn hard alive world want keep going want keep fighting wish would end could peace ever asked life wa love ever wanted wa love world took request turned back form million dagger heart bear want die much pain | 1 | Suicide |
| 1893 | eli sbt low iq twice tested time got iq around 84 85 somehow entirely awful school wa able b student without much studying actually started studying started get grade 90 even 100 even though skimmed exemplary grade barely retained learned nobody around belief stupid grade go poor vocabulary overall especially spoken word cannot basic mental math still count finger nineteen year old low income uneducated family wa innately disadvantaged birth also mom smoke weed cigarette throughout pregnancy 8th month pregnancy detached placenta preform emergency c section likely brain damage oxygen deprivation feel incredibly empty every moment awake favorite time day go sleep night night always go quickly lately spending everyday sleeping much joy dream want least average intelligence least unbearable aware every moment dumb ever dumb always aspired leave financial hole everyday pass reminds probably never happen point feel like stuck rock hard place terrified leave behind family pet know continu... | 1 | Suicide |
| 1894 | gay h ovum feel useless im useless useless | 1 | Suicide |
| 1895 | ev e wish wa way could non existence regret entire life know take last breath matter hand regret life stopping know would probably kill dad want daughter feel like would better without would want mom like doormat everyone go people tell whenever ask help get thrown back face point anymore sacrifice enough say even though life anymore even room breathe love selfish selfish wish hd jsut hour week something productive get rid noise let breathe already asked help fault fault making time one seems understand matter much explain much ask help breaking anyone understand doormat doormat even stand try doormat shit get stomped even deeper every fiber feel like agony agony agony regret everything fucking miserable miserable miserable wish never existed wish never made tie life awful would anyone ever sentient creature life fuckkng horrible long living like agony terrible mother sacrifice enough wish never existed never knew anyone one ever knew wish wa easy disappearing thin air trace memory... | 1 | Suicide |
| 1896 | pon k ichi want die think want feel pain want feel something death way get sorry posting much today anyways im torn rn dont know right | 1 | Suicide |
| megatext_clean | is_suicide | category | parsed | |
|---|---|---|---|---|
| 1892 | big pete 543 hard get bed every morning right want die damn hard alive world want keep going want keep fighting wish would end could peace ever asked life wa love ever wanted wa love world took request turned back form million dagger heart bear want die much pain | 1 | Suicide | (big, pete, 543, hard, get, bed, every, morning, right, want, die, damn, hard, alive, world, want, keep, going, want, keep, fighting, wish, would, end, could, peace, ever, asked, life, wa, love, ever, wanted, wa, love, world, took, request, turned, back, form, million, dagger, heart, bear, want, die, much, pain) |
| 1893 | eli sbt low iq twice tested time got iq around 84 85 somehow entirely awful school wa able b student without much studying actually started studying started get grade 90 even 100 even though skimmed exemplary grade barely retained learned nobody around belief stupid grade go poor vocabulary overall especially spoken word cannot basic mental math still count finger nineteen year old low income uneducated family wa innately disadvantaged birth also mom smoke weed cigarette throughout pregnancy 8th month pregnancy detached placenta preform emergency c section likely brain damage oxygen deprivation feel incredibly empty every moment awake favorite time day go sleep night night always go quickly lately spending everyday sleeping much joy dream want least average intelligence least unbearable aware every moment dumb ever dumb always aspired leave financial hole everyday pass reminds probably never happen point feel like stuck rock hard place terrified leave behind family pet know continu... | 1 | Suicide | (eli, sbt, low, iq, twice, tested, time, got, iq, around, 84, 85, somehow, entirely, awful, school, wa, able, b, student, without, much, studying, actually, started, studying, started, get, grade, 90, even, 100, even, though, skimmed, exemplary, grade, barely, retained, learned, nobody, around, belief, stupid, grade, go, poor, vocabulary, overall, especially, spoken, word, cannot, basic, mental, math, still, count, finger, nineteen, year, old, low, income, uneducated, family, wa, innately, disadvantaged, birth, also, mom, smoke, weed, cigarette, throughout, pregnancy, 8th, month, pregnancy, detached, placenta, preform, emergency, c, section, likely, brain, damage, oxygen, deprivation, feel, incredibly, empty, every, moment, awake, favorite, time, day, ...) |
| 1894 | gay h ovum feel useless im useless useless | 1 | Suicide | (gay, h, ovum, feel, useless, im, useless, useless) |
| 1895 | ev e wish wa way could non existence regret entire life know take last breath matter hand regret life stopping know would probably kill dad want daughter feel like would better without would want mom like doormat everyone go people tell whenever ask help get thrown back face point anymore sacrifice enough say even though life anymore even room breathe love selfish selfish wish hd jsut hour week something productive get rid noise let breathe already asked help fault fault making time one seems understand matter much explain much ask help breaking anyone understand doormat doormat even stand try doormat shit get stomped even deeper every fiber feel like agony agony agony regret everything fucking miserable miserable miserable wish never existed wish never made tie life awful would anyone ever sentient creature life fuckkng horrible long living like agony terrible mother sacrifice enough wish never existed never knew anyone one ever knew wish wa easy disappearing thin air trace memory... | 1 | Suicide | (ev, e, wish, wa, way, could, non, existence, regret, entire, life, know, take, last, breath, matter, hand, regret, life, stopping, know, would, probably, kill, dad, want, daughter, feel, like, would, better, without, would, want, mom, like, doormat, everyone, go, people, tell, whenever, ask, help, get, thrown, back, face, point, anymore, sacrifice, enough, say, even, though, life, anymore, even, room, breathe, love, selfish, selfish, wish, hd, jsut, hour, week, something, productive, get, rid, noise, let, breathe, already, asked, help, fault, fault, making, time, one, seems, understand, matter, much, explain, much, ask, help, breaking, anyone, understand, doormat, doormat, even, stand, try, doormat, ...) |
| 1896 | pon k ichi want die think want feel pain want feel something death way get sorry posting much today anyways im torn rn dont know right | 1 | Suicide | (pon, k, ichi, want, die, think, want, feel, pain, want, feel, something, death, way, get, sorry, posting, much, today, anyways, im, torn, rn, dont, know, right) |
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1897 entries, 0 to 1896 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 title 1897 non-null object 1 selftext 1897 non-null object 2 author 1897 non-null object 3 num_comments 1897 non-null int64 4 is_suicide 1897 non-null int64 5 url 1897 non-null object 6 selftext_clean 1897 non-null object 7 title_clean 1897 non-null object 8 author_clean 1897 non-null object 9 selftext_length 1897 non-null int64 10 title_length 1897 non-null int64 11 megatext_clean 1897 non-null object dtypes: int64(4), object(8) memory usage: 178.0+ KB